1. Technical Field
The present invention relates to a video and audio multiplex transmission system, more particularly relates to a system for coding audio data and video data and multiplexing and transmitting the same.
In recent years, there has been a boom in communications transmitting both audio data and video data, such as with video conferences and television-equipped telephones. In such a multiplex communications system, it is becoming important to performing the coding and decoding of the two at a high efficiency and to maintain a balance between the audio data and video data.
2. Description of the Related Art
In a video and audio coding system known in the past, which transmits coded audio data multiplexed with video coded data and other control data, in the case of a transmission speed of a particularly low bit rate such as in a 2.times.B system which uses two 64 kb/s lines, the 2B system which uses one 128 kb/s line, or the B system which uses one 64 kb/s line, the ratio of transmission between the audio and video signals is in general fixed to about 1:1 (for example, the case where the audio coding speed is 56 kb/s and the video coding speed is 64 kb/s), about 1:3 (for example, the case where the audio coding speed is 32 kb/s and the video coding speed is 96 kb/s), or about 1:7 (for example, the case where the audio coding speed is 16 kb/s and the video coding speed is 112 kb/s).
However, the ratio of the data density between the audio signal and video signal is inherently different by several hundred-fold. Despite this, the data is transmitted by the above ratios. Therefore, in the case of a transmission ratio of 1:1, the amount of video data transmitted per unit time becomes small, inviting deterioration of the quality of the reproduction of moving images. On the other hand, in the case of a transmission ratio of 1:7, the audio quality becomes poorer. In particular, in the latter case, when coding a picture with little movement, unnecessary bits (fill bits) are added for matching of the transmission bit rate so as to meet with the transmission capacity.
As mentioned above, in the conventional system, there has been the problem of a sacrifice of one of the video quality and audio quality.
In the video coding unit used in such a video and audio multiplexing system, the redundancy of the video data is reduced by interframe predictive coding, intraframe predictive coding, motion compensation predictive coding, variable word length coding, and other coding techniques, but the amount of data for video signals is far greater than for audio signals, so a delay occurs with respect to the coding and decoding processing times of audio signals.
Therefore, it was not possible to secure synchronization between the picture and sound at the receiving side (lip sync), and an unnatural reproduction with mismatched picture and sound resulted.
Therefore, the technique was used of finding in advance the average of the delay times of processing of video signals with respect to the sound and, based on this as a reference, causing a fixed delay time determined during the audio processing of the system. However, there was a problem in that the actual delay time changed depending on the content of the video data and therefore the unnaturalness could not be sufficiently improved.
Normally, also, the amount of the video data is far greater than that of the audio data, so not all of the video data is actually transmitted. That is, the portion which is not transmitted is ignored and the video data is sent with lapses. Due to this, the picture reproduced at the receiver side appears rough in movement. This is because picture quality is less important than real time transmission.
On the other hand, with respect to audio signals, there are silent periods in sound. Despite this, 16 kps/s worth or 56 kps/s worth of transmission capacity is secured. Therefore, it is desirable to use the silent periods in audio signals for the video signals.